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IMAGE PICKUP METHOD AND APPARATUS 

BACKGROUND OF THE INVENTION 
Field of -tlie Inven-tion 
5 The present invention relates to an image pickup 

method and an image pickup apparatus capable of 
broadening an apparent dynamic range of a video signal. 
Related Background Art 

Image pickup devices are used widely as an 

10 image pickup unit of VTRs with a built-in camera, still 
video cameras and the like- An image pickup apparatus 
using an image pickup tube of a video camera or a solid 
image pickup element has a dynamic range narrower than 
that of silver- salt photographing systems, and is 

15 therefore associated with some problems of "crushed 

white" with lost tonal levels in a bright portion and 
"crushed black" with lost tonal levels in a dark 
portion, under rear light or in other cases. 

If such a case occurs with, for example, a video 

20 camera, a user increases an exposure amount of a main 

object with "crushed black" by manually controlling the 
iris or operating upon a rear light correction button 
to open the iris by about two steps. 

Even if a proper exposure amount is obtained for 

25 the main object through rear light correction, the 

background has "crushed white" and becomes only white. 
Therefore, with a conventional image pickup apparatus. 



proper adjustment only of the exposure amount for a 
main object does not solve a problem of a narrow 
dynamic range of the image pickup apparatus. 

In order to solve the problem of "crushed white" 
5 and "crushed black", in an image pickup apparatus which 
uses a line scanner or the like for the conversion of a 
still image into electrical signals, a single image is 
synthesized from a plurality of images of the same 
object picked up at different exposure amounts. 

10 In the case of such an image pickup apparatus 

taking only a still image, the positional relation 
between an object and the apparatus is fixed. 
Therefore, there is no shift between images although 
they are picked up at different timings, so that there 

15 is no practical problem even if a single image is 
synthesized from a plurality of images. 

However, in the case of an image pickup apparatus 
such as a video camera or a still video camera, the 
above-described image pickup method cannot be applied 

20 reliably because the positional relation between an 

object and the apparatus may change minutely because of 
hand vibration or the like. 

In such a case, the position of each of a 
plurality of images of the same object picked up at 

25 different timings has a small shift. Therefore, a 
single image synthesized from these images has the 
object with an image shift such as double or triple 



objects. This becomes a serious issue for recent 
compact and light video cameras which are likely to 
have hand vibration. 

SUMMARY OF THE INVENTION 

In consideration of the above-described problems, 
an object of the present invention is to provide an 
image pickup method and an image pickup apparatus 
capable of picking up an image with a broad substantial 
dynamic range without the image shift, even with video 
cameras or still video cameras which are likely to be 
subject to image shifts. 

According to one aspect of the present invention, 
there is provided an image pickup method of increasing 
an apparent dynamic range of a video signal by 
synthesizing a single image from a plurality of images 
sequentially picked up at different exposure amounts, 
wherein a motion vector of a video signal is detected, 
and if the detection result indicates that the motion 
vector is larger than a predetermined threshold value, 
image synthesization is not performed. 

According to another aspect of the present 
invention, there is provided an image pickup method of 
increasing an apparent dynamic range of a video signal 
by synthesizing a single image from a plurality of 
images sequentially picked up at different exposure 
amounts, comprising: a motion vector detecting step of 



detecting a motion vector between corresponding pixels 
in the plurality of images; a vector difference 
detecting step of detecting a difference between a 
motion vector detected at the motion vector detecting 
step and a motion vector between the plurality of 
images; a comparison step of comparing the detection 
result of the vector difference detecting step with a 
predetermined threshold value; and a synthesization 
inhibiting step of inhibiting the image synthesization 
if the comparison result at the motion vector 
comparison step is larger than the predetermined 
threshold value. 

According to another aspect of the present 
invention, there is provided an image pickup method of 
increasing an apparent dynamic range of a video signal 
by synthesizing a single image from a plurality of 
images sequentially picked up at different exposure 
amounts, comprising: a motion vector detecting step of 
detecting a motion vector between corresponding pixels 
in the plurality of images; a vector difference 
detecting step of detecting a difference between a 
motion vector detected at the motion vector detecting 
step and a motion vector between the plurality of 
images; a comparison step of comparing the detection 
result of the vector difference detecting step with a 
predetermined threshold value; a coordinate converting 
step of performing a coordinate conversion of the 
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plurali-ty of images in correspondence with, an image 
Shift caused by a time difference when the plurality of 
images are picked up, if the comparison result at the 
motion vector comparison step is smaller than the 
5 predetermined threshold value; and an image 

synthesizing step of synthesizing the plurality of 
images with the image shift corrected at the coordinate 
converting step, into a single image. 

According to another aspect of the invention, 

10 there is provided an image pickup apparatus for 

increasing an apparent dynamic range of a video signal 
by synthesizing a single image from a plurality of 
images sequentially picked up at different exposure 
amounts, comprising: motion vector detecting means for 

15 detecting a motion vector of the video signal; 

comparison means for comparing the detection result by 
the vector difference detecting means with a 
predetermined threshold value; and synthesization 
inhibiting means for inhibiting the image 

20 synthesization if the comparison result by the 

comparison means is larger than the predetermined 
threshold value. 

According to another aspect of the invention, 
there is provided an image pickup apparatus for 

25 increasing an apparent dynamic range of a video signal 
by synthesizing a single image from a plurality of 
images sequentially picked up at different exposure 



amounts, comprising: mo-tion vector de-fcecting means for 
detecting a motion vector between corresponding pixels 
in the plurality of images; vector difference detecting 
means for detecting a difference between a motion 
vector detected by the motion vector detecting means 
and a motion vector between the plurality of images; 
comparison means for comparing the detection result by 
the vector difference detecting means with a 
predetermined threshold value; coordinate converting 
means for performing a coordinate conversion of the 
plurality of images in correspondence with an image 
shift caused by a time difference when the plurality of 
images are picked up, if the comparison result by the 
motion vector comparison means is smaller than the 
predetermined threshold value; and image synthesizing 
means for synthesizing the plurality of images with the 
image shift corrected by the coordinate converting 
means, into a single image - 

The invention provides the above -described 
technical aspects. Therefore, in increasing an 
apparent dynamic range of a video signal by 
synthesizing a single image from a plurality of images 
sequentially picked up at different exposure amounts, 
if a difference between a motion vector between pixels 
in a plurality of images and a motion vector between 
images is larger than a predetermined threshold value, 
the pixel is not used for synthesization. Even a video 



signal having an Image shift caused by a mo-tion of an 
object itself or hand vibration can be prevented from 
forming a double or triple images of an object. 

Other objects, features and advantages of the 
5 invention will become apparent from the following 
detailed description taken in conjunction with the 
accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 
10 Fig. 1 is a block diagram showing the structure of 

a VTR with a built-in camera embodying the present 
invention. 

Fig. 2 is a block diagram showing the specific 
structure of a camera unit of VTR shown in Fig. 1. 
15 Fig. 3 is a timing chart illustrating an operation 

of an image pickup device. 

Fig. 4 is a diagram explaining a concept of image 
processing according to an embodiment of the invention. 

Fig. 5 is a block diagram showing the specific 
20 structure of an operation (computing) circuit of VTR 
shown in Fig. 1. 

Figs. 6A and 6B are diagrams explaining a method 
of determining threshold levels to be used for the 
discrimination between "crushed white" and "crushed 
25 black". 

Figs. 7 A, 7B, 7C and 7D are diagrams explaining 
tonal level characteristics. 
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Fig. 8 is a timing chart illustrating the 
operation of an image pickup device. 

Fig. 9 is a block diagram showing the partial 
structure of the camera unit. 
5 Fig. 10 is a block diagram showing an example of a 

control circuit of VTR shown in Fig. 1. 

Figs. IIA, IIB and IIC are diagrams explaining a 
concept of image shifts according to another 
embodiment . 

10 Fig. 12 is a block diagram showing the specific 

structure of VTR of the embodiment. 

Fig. 13 is a timing chart illustrating the 
operation of VTR of the embodiment. 



15 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

An image pickup method and an image pickup 
apparatus according to preferred embodiments will be 
described in detail with reference to the accompanying 
drawings . 

20 Fig. 1 is a block diagram showing the overall 

structure of a VTR with a built-in camera embodying the 

present invention. 

In Fig. 1, reference numeral 100 denotes a camera 

unit, reference numeral 200 denotes a processing unit, 
25 and reference numeral 300 denotes a recording unit. In 

the camera unit 100, light incident upon an optical 
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system 101 is controlled in amount by an iris 102 and 
focussed on an image pickup device (element) 103. 

The image pickup device 103 is made of 
semiconductor elements such as MOSs and CCDs. An 
5 optical image focussed on the focal plane of the image 
pickup device 103 is converted into electrical video 
signals to generate a video signal to be output. 

A focus drive circuit 107, an iris drive circuit 
106 and an image pickup device drive circuit 105 for 

10 controlling a light accumulation time or the like drive 
the optical system 101, iris 102, and image pickup 
device 103, under the control of a camera control 
circuit 108. A camera signal processing circuit 104 
performs a gamma correction process or other known 

15 processes, similar to a signal processing circuit of a 
general video camera. 

The video signal output from the camera unit 100 
is an analog signal which is supplied to the processing 
unit 100 and converted into a digital signal by an A/D 

20 converter 201 . Pixel data of the converted digital 
signal is then converted by an operation ( computing ) 
circuit 202 in a manner to be described later. 
Thereafter, the converted pixel data is converted back 
into an analog signal by a D/A converter 203 and 

25 supplied to the recording unit 300. 

Reference numeral 204 denotes an image memory 
which is used by the operation circuit 202 and a motion 



vector detection circuit 206 for data calculation. 
Reference numeral 205 denotes a memory control circuit 
for controlling the operation of the image memory 204. 
The memory control circuit 205 outputs read/write 
5 address signals of the image memory 204 in response to 

a timing signal supplied from the camera control 
circuit 108 of the camera unit 100. 

The memory control circuit 205 also outputs an 
address signal for frame position alignment, according 
10 to an output from the motion vector detection circuit 
206, when images are synthesized by correcting an 
image shift. 

A motion vector comparison circuit 207 compares a 
motion vector of each pixel detected by the motion 

15 vector detection circuit 203 with a motion vector 

between images calculated from the motion vector of 
each pixel. If a difference of the motion vector of 
each pixel from the motion vector between images is 
equal to or larger than a predetermined threshold 

20 value, position information of each pixel is supplied 
to the operation circuit 202 to inhibit synthesization 
of pixels. 

The operation circuit 202 synthesizes read common 
signals and performs enlarging/interpolation in order 
25 to convert the common signals into signals of the same 
type as original input signals. In this manner, the 
operation circuit 202 outputs a video signal of even a 
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moving object:, with a synthesized tonal level without 
an image shift. 

The video signal output from the operation circuit 
202 is converted by the D/A converter 203 into an 
5 analog signal and recorded in VTR ( video tape recorder ) 
301. If VTR 301 is a digital recording type, the D/A 
converter 203 is not necessary. 

Next, the details of the operation of the image 
pickup element 103 will be described. Fig. 2 is a 
10 block diagram showing the more detailed structure of 
the camera unit 100, and Fig. 3 is a timing chart 
illustrating the operation of the camera unit 100 
assuming that NTSC signals are used. 

Referring to Fig. 3, a field index signal (Fl) is 
15 a signal used for discriminating between odd and even 
fields constituting one frame. 

A Vblk signal is a vertical blanking signal, the 
period with H (high) of this signal corresponding to an 
effective frame and the period L (low) corresponding to 
20 a vertical blanking period. 

A Tpyj^gg signal controls a charge accumulation time 
of the image pickup device 103. This Tp^j^^^ signal is a 
pulse signal used for reading a pixel to be output to a 
vertical transfer CCD, if the image pickup device is a 
25 CCD image pickup device. 

An iris gate signal is a signal designating which 
of a 1/1000 sec. accumulation signal and a 1/60 sec. 
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accumulation signal is used as a standard video signal 
for au-tomatic exposure to be described later. 

In the example shown in Fig. 3, during the first 
blanking period, the 1/1000 sec. charge accumulation is 
5 performed, and during the next effective period, the 

1/1000 sec. accumulation signal is output. Immediately 
after the 1/1000 sec. accumulation period, the 
substantial 60 sec. charge accumulation is performed, 
and during the next field effective period, the 1/60 

10 sec. accumulation signal is output- In this manner, 
signals of difference light amounts (1/1000 sec. and 
1/60 sec. ) are alternately output for each field. 

In Fig. 2, reference numeral 20 denotes a known AE 
control circuit which receives a signal (e.g., video 

15 signal) from the camera signal processing circuit 104 

via the A/D converter 201 and calculates and outputs an 
exposure control signal. Reference numeral 22 denotes 
a known AF control circuit which outputs a focus 
control signal- Reference numeral 24 denotes a 1/2 

20 frequency division circuit for lowering the frequency 
of the vertical blanking signal Vb^k by a half. 

Reference numerals 26 and 27 denote sample-hold 
circuits, reference numeral 28 denotes an inverter, and 
reference numerals 29 and 30 denote switches which 

25 select either an output of the 1/2 frequency division 
circuit 24 or an output of the 1/2 frequency division 
circuit 24 inverted by the inverter 28, in order to 
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determine a sampling timing. Outputs of the sample- 
hold circuits 26 and 27 are applied to the iris drive 
circuit 106 and focus drive circuit 107, respectively, 
to thereby perform an automatic exposure control, an 
5 automatic focus control and the like according to those 
outputs . 

In this embodiment, a combination of 1/1000 sec. 
and 1/60 sec. is used, which corresponds to a light 
amount change of about four steps ( sixteenf old ) . 

10 Therefore, for example, in the case of a camera using a 
CCD image pickup device, for example, if an exposure 
control of a main object is performed by using as a 
reference the 1/60 sec. accumulation time in the even 
field, it happens frequently that the background of the 

15 even field is likely to have the "crushed white", 

whereas the main object in the odd field with the light 
amount lowered by four steps is likely to have the 
"crushed black". This example assumes the case wherein 
proper exposure is set to the background for rear light 

20 correction, and in accordance with the scene at that 
time, the accumulation time may be set to a value 
different from 1/1000 sec. 

As different from the embodiment CCD, a CCD having 
a high speed shutter function of a VOD type has been 

25 recently used. This type drains unnecessary charges in 
the vertical direction of a CCD substrate, and can set 
the shutter speed very finely- If this CCD is used. 
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the camera drive circuit 105 can set an optimum shutter 
speed which is judged by the AE control circuit 20 of 
the control circuit 108 in accordance with a brightness 
difference between a main object and a background 
5 object. 

As described earlier, in this embodiment, the 
image quality is improved by positively utilizing the 
"crushed white" and "crushed black" in each field. 
That is, a portion with the "crushed white" or "crushed 

10 black" is replaced by a corresponding portion in the 

other field (since exposure is different, the "crushed 
black" or "crushed white" is not being formed in the 
other field), and video signals of both the fields are 
synthesized to obtain a final video signal. 

15 The above fundamental approach will be explained 

with reference to Fig. 4. In Fig. 4, a main object is 
schematically shown as a rectangle with a vertically 
long side. A through image (T) shown in Fig. 4 
corresponds to an image formed directly by an output of 

20 the image pickup device 103. A memory image (M) 

corresponds to an image formed by a preceding field 
image signal once stored in the image memory 204. The 
through image has the main object with the "crushed 
black" by rear light at each odd field, whereas it has 

25 the background with the "crushed white" at each even 
field. 
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Since the memory image is formed by a signal 
delayed by one field, the "crushed white" and "crushed 
black" are formed in the fields different from those of 
the through images. Therefore, if the through and 
5 memory images are properly combined, an image of a good 
quality without the "crushed white" and "crushed black" 
can be obtained. Namely, signals of the through and 
memory images are compared with predetermined threshold 
values at each field to discriminate between the 

10 "crushed white" and "crushed black" of each pixel, by 
setting "1" if the signal of each pixel is larger than 
the threshold value, and "0" if smaller. 

Figs. 6A and 6B show the relation between 
threshold values, pixel brightness levels, and fields. 

15 The abscissa of Fig. 6A indicates the brightness level, 
and the ordinate indicates an occurrence frequency of 
each brightness level in one frame. As shown in Fig. 
6A, a first threshold value Thl is set so that the 
"crushed black" can be discriminated, whereas a second 

20 threshold value Th2 is set so that the "crushed white" 
can be discriminated. That is, a brightness level 
equal to or higher than the first threshold value Thl 
is judged as the "crushed black", whereas a brightness 
level equal to or higher than the second threshold 

25 value Th2 is judged as the "crushed white". 

Fig. 6B shows the relation between each field and 
a corresponding threshold value. Since the "crushed 
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white" and "crushed black" appear alternately in the 
odd and even fields, the threshold values used for the 
discrimination are also changed for each field. 

It is possible therefore to Judge which pixel in 
which field has the "crushed black" or "crushed white". 
By using the judgement results, a pixel signal having a 
proper exposure amount can be selected from either the 
through or memory image. For example, in the odd 
field, the through image is selected for a pixel having 
a logical product "1" of a judgement A and a judgement 
B, and the memory image is selected for a pixel having 
the logical product "0". In the even field, the 
opposite relation as above is set- In this manner, 
selection flags shown in Fig. 4 can be obtained. 

Images synthesized in accordance with the 
selection flags are shown at the bottom row of Fig. 4. 
As shown, moving images sufficient for practical use 
can be obtained, by suppressing the influence of the 
time axis shift of each image assuming that the main 
object is moving at a constant speed. 

Fig. 5 is a block diagram showing the details of a 
circuit portion in the operation circuit 202 of the 
processing circuit 200, the circuit portion being used 
for the comparison with the first and second threshold 
values Thl and Th2 and for the generation of the 
selection flags. A threshold value switch control 
signal alternately changes between "H" and "L" at each 
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field, similar to an FI signal. The threshold value 
switch control signal is applied to a first threshold 
value generation circuit 53, and via an inverter 51 to 
a second threshold value generation circuit 52. 
5 In accordance with the threshold value switch 

control signal, the first and second threshold value 
generation circuits generate the first or second 
threshold value Thl or Th2 having the relation shown in 
Fig. 6B. Comparison circuits 54 and 55 compare the 

10 memor-y and through images with the threshold values 

supplied from the threshold value generation circuits 
52 and 53, and output judgement signals A and B. 

An AND gate 56 outputs a selection flag signal as 
the logical product of the Judgement signals A and B. 

15 A switch 57 selects either the memory or through 

image in accordance with the selection flag signal. 

Figs, 7A to 7D are diagrams explaining tonal level 
characteristics- A solid line in Fig. 7A shows a 
characteristic of a general video camera. Up to a 100 

20 % input, the input/output characteristic is linear. 

For an input {100 to 400 %), a gentle slope called a 
knee characteristic appears- A transition point 
between the linear characteristic portion and knee 
characteristic portion is represented by PI. For a 

25 high speed shutter, this transition point moves to P2 . 

It is assumed that PI is 1/60 sec and P2 is 1/250 sec. 
with an exposure change of two steps. 



As described earlier, if the exposure amount is 
four steps, the shutter speed change is represented by 
the relation between (1) and (5) shown in Fig. 7D. (1) 
in Fig. 7D corresponds to 1/60 sec, (2) corresponds to 
1/125 sec, (3) corresponds to 1/250 sec, (4) 
corresponds to 1/500 sec, and (5) corresponds to 
1/1000 sec A characteristic with a desired curve can 
be obtained by using two characteristic lines having 
different slopes. 

Fig. 7B shows examples of synthesized 
characteristic lines in which different threshold 
values are used for the discrimination between the 
"crushed white" and "crushed black" . 

Fig. 7C shows characteristic lines, (1) outputting 
an addition average of corresponding two pixel signals, 

(2) outputting a selected one of two pixel signals, and 

(3) outputting an addition average with predetermined 
coef f icient s . 

In the above example, a time resolution is 
substantially 30 images per second which is almost 
equal to a frame storage CCD image pickup device. In 
order to realize a time resolution almost equal to a 
field storage CCD image pickup device, an example of 
picking up two images in one field, will be described. 
A changed circuit portion is shown in Fig. 9, and the 
timing chart is shown in Fig . 8 . 
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Principally, signals are read from the image 
pickup device 103 at a speed higher than a general 
video rate, and the read signals are converted in a 
time axis into a normal rate. First and second field 
5 memories 90 and 91 each have a memory capacity of one 
field image information. The first field memory 90 
delays a 1/1000 sec. accumulation signal in order to 
synchronize it with a 1/120 sec. read signal. The 
second field memory 91 performs a twofold time 
10 expansion in order to change the 1/120 sec. video 
signal to an NTSC signal of 1/60 sec. 

Rows (a) to (d) shown in Fig. 9 correspond to 
signals (a) to (d) shown in Fig. 8. The image pickup 
device 103 capable of performing the above operation 
15 may be a MOS solid image pickup device of an XY 
addressing type. 

Next, another example of the details of the 
control circuit 108 will be described with reference to 
Fig. 10. A master clock generator 40 generates an 
20 internal master clock to be used by the control circuit 
108, in accordance with an external reference signal - 

A clock generator 41 for a 1/1000 sec. shutter 
generates a high speed clock from the master clock 
supplied from the master clock generator 40. A clock 
25 generator 42 for a 1/60 sec. shutter generates a low 
speed clock from the master clock supplied from the 
master clock generator 40. 
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A switch 45 changes its contact point alternately 
at each field to alternately apply the outputs of the 
clock generators 41 and 42 to the drive circuit 105. 

An AE control signal generator 43 generates an AE 
5 control signal for the iris control, in accordance with 
the video signal supplied from the camera signal 
processing circuit 104. A control signal holding 
circuit 44 holds the AE control signal during one field 
per-iod. 

10 A switch 46 changes its contact point alternately 

at each field to alternately apply the AE control 
signal of the AE control signal generator 43 and the AE 
control signal held by the control signal holding 
circuit, to the iris control circuit 106. A switch 

15 signal generator 47 controls the switching of the 

switches 45 and 46 which are synchronously switched - 
In the above example, the clock generators for 
generating high and low speed clocks are provided, and 
the high and low speed clocks are switched in response 

20 to a signal of the switch signal generator output at 
each field. Accordingly, the circuit structure and 
operation can be simplified which are particularly 
suitable for moving images. 

In the above description, images are formed at 

25 different exposure amounts by changing the shutter 

speed. If a high speed iris function is available, the 
iris may be changed at high speed. Alternately, a 
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light reducing filter using a spatial light modulating 
device such as PLZT may be electrically controlled. 

As described above, various calculations may be 
made in this embodiment. An example of image 
5 synthesization to be performed with a simple process 

and more effective simple addition calculation will be 
described. 

Figs. IIA to lie are conceptual diagrams 
explaining an image shift of an object in the two 
10 images (odd and even field images) relative to a 
display screen ( image pickup device 103 ) . 

It is assumed that the upper left in the common 
area of the even and odd images is a memory read start 
point P. Then, a point Pqup shown in Fig. IIB is the 
15 image memory read start point for the odd image, and a 
point Peven shown in Fig. IIC is the image memory read 
start point for the even image. By using these start 
points, video signals with no image shift can be 
generated for the common area of the even and odd 
20 images . 

A specific example of the structure shown in Fig. 
1 applied to this embodiment is shown in Fig. 12. 

Referring to Fig. 12, an input video signal is 
converted into a digital signal by the A/D converter 
25 201, and supplied to a first field memory 204a and a 
motion detection circuit 206. 
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An output of the first field memory 204a is 
supplied to another input terminal of the motion vector 
detection circuit 206. The motion vector detection 
circuit 206 detects motion information of each pixel 
5 from two video signals having one field time delay, and 
in accordance with this motion information, generates 
an image shift correction signal. This image shift 
correction signal is supplied to the memory control 
circuit 205 which controls the first field memory 204a 

10 in accordance with the image shift correction signal. 

The image shift correction signal generated by the 
motion vector detection circuit 206 is supplied also to 
the motion vector comparison circuit 207 . The motion 
vector comparison circuit 207 is constituted of a 

15 threshold generation circuit 61, a comparison circuit 
62, an address conversion circuit 63, and the like. 
The image shift information output from the motion 
vector detection circuit 206 is supplied to the 
threshold generation circuit 61 and comparison circuit 

20 62. The threshold generation circuit 61 generates a 
threshold value to be used for the judgement of an 
object, and supplies it to the comparison circuit 62. 

The comparison circuit compares the threshold 
value with the motion information of each pixel to 

25 judge whether the motion of each pixel is caused by 
hand vibration or by some other motion. In other 
words, it is judged whether the motion of each pixel is 
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caused by the motion of the object itself, and an 
address of a pixel which was judged as having a motion 
other -than by hand vibra-tion, is supplied to the 
address conversion circuit 63. 

In accordance with interpolation information 
supplied from an enlarging/interpolation circuit 202a, 
the address conversion circuit 63 converts an address 
of the pixel which was judged as having a motion other 
than by hand vibration, into an address after 
enlarging/ interpolation, generates a selection signal 
for inhibiting the pixel to be synthesized and 
selecting the pixel in the field with a proper 
exposure, and supplies the selection signal to a next 
stage selector 202c. 

The video signal of the common area shown in Fig. 
IIA is output from the first field memory 204a and 
converted into a signal of the original type (e.g., 
NTSC signal) same as the input signal. 

The smaller the common area, the larger the 
enlarging factor becomes. Regardless of what enlarging 
factor is used, the video signal is converted into a 
signal of the original type and stored via an adder 
202b into a second field memory 204b. 

Assuming that the video signal stored in the 
second field memory 204b is of the odd field, the video 
signal of the next even field is output from the 
enlarging/interpolation circuit 202a and added to the 
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corresponding video signal by the adder 202b for each 
pixel. 

The addition signal is supplied via the terminal A 

of the selector 202c and via the EVEN terminal of 
5 another selector 207 to the D/A converter 203. The 

addition signal is also supplied via the terminal A of 

the selector 202c to the second field memory 204b. 

During the next odd field, new odd field information is 

stored in the second field memory 204b while the same 
10 video signal is supplied to the D/A converter 203 via 

the ODD terminal of the selector 207 through a read- 

modify-write operation. 

A switch 208 is open for the odd field and close 

for the even field. While the new odd field 
15 information is written in the second field memory, the 

adder 202b is made through by connecting to the 

terminal E . 

The terminal A of the selector 202c is supplied 
with the addition signal, the terminal O is supplied 

20 with the odd field information output from the second 
field memory 204b, and the terminal E is supplied with 
the even field information. If the selection signal 
from the motion vector comparison circuit 207 indicates 
that a pixel has a motion caused by hand vibration, the 

25 addition signal is selected, whereas if the selection 
signal indicates that a pixel has a motion other than 
by hand vibration, a video signal of either the even or 
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odd field having a proper exposure is selected. The 
selec-ted signal is supplied to the next stage selector 
207 and second field memory 204b. A video signal 
generated in the above manner is output from the D/A 
converter 203 in the form of analog signal same as the 
original input signal . 

The operation of the above-described processes is 
illustrated in the timing chart of Fig. 13. 

In the timing chart shown in Fig. 13, a signal 
indicated at A is a discrimination signal between the 
odd and even fields and corresponds to an output timing 
of the enlarging/interpolation circuit 202a. 

A signal indicated at B shows an open/close timing 
of the switch 208- The function of the adder 202b 
changes with this open/close of the switch 208. If the 
switch 208 is open, the signal output from the 
enlarging/interpolation circuit 202a passes through the 
adder 202b, whereas if the switch 208 is close, the 
signal is added by the adder 202b. 

A signal indicated at C shows a write timing of 
the second field memory 204b. During the odd field 
indicated by the signal A, the odd field image signal 
is written in the second field memory, whereas during 
the even field indicated by the signal A, the video 
signal selected by the selector 202c according to the 
read-modify-write operation is written again in the 
second field memory 204b at the same addresses. 
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A signal indicated at D shows a switching timing 
of the selector 207. Image information synthesized 
from the odd field image information during a period tl 
and the even field image information during a period t2 
is output during a period t2, and the same synthesized 
image information is again read and output during a 
period t3- 

In synthesizing an image, if it is judged that a 
pixel has a motion other than by hand vibration, a 
video signal of either the even or odd field having a 
proper exposure is selected, so that a time axis shift 
of the image may be feared. However, this does not 
pose any practical problem, as described with an 
example of a calculation process for a synthesized 
image shown in Fig. 4. 

As described previously, in this embodiment, each 
process is performed in the unit of two fields. A line 
interpolation process may be interchanged between the 
odd and even fields to make compatible with interlacing 
and reduce so-called field interference. In order to 
reduce this field interference, a line interpolation 
circuit is inserted at a point Q between the second 
field memory 204b and selector 207. 

The invention is applicable to any modification or 
alteration of the above-described embodiment without 
departing from the scope and spirit thereof. 



- 27 - 



The present invention may be applied to a system 
constituted by a plurality of apparatuses (e.g., a host 
computer, interface units, a reader, a printer, and the 
like) or to a system constituted by a single apparatus. 

The scope of the invention also includes a system 
or apparatus whose computer (CPU or MPU) runs to 
operate various devices connected thereto in accordance 
with software program codes supplied to the system or 
apparatus so as to realize the functions of the above 
embodiments . 

In this case, the software program codes 
themselves realize the embodiment functions. 
Therefore, the program code themselves and means for 
supplying such program codes to a computer, e.g., a 
storage medium storing such program codes, constitute 
the present invention. 

The storage medium storing such program codes may 
be a floppy disk, a hard disk, an optical disk, a 
magneto-optical disk, a CD-ROM, a magnetic tape, a non- 
volatile memory card, and a ROM. 

Obviously, such program codes are other types of 
embodiments of this invention, not only for the case 
wherein the embodiment functions are realized by 
executing the program codes supplied to the computer 
but also for the case wherein the embodiment functions 
are realized by the program codes used with an OS 
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( operating system ) on which the computer runs or with 
other various types of application software. 

Furthermore, the scope of the invention also 
includes obviously the case wherein in accordance with 
5 the program codes stored in a memory of a function 
expansion board or unit connected to the computer 
supplied with the program codes, a CPU or the like of 
the function board or unit executes part or the whole 
of the actual tasks for realizing the embodiment 

10 functions. 

As described so far, according to the present 
invention, a plurality of images are sequentially 
picked up at different exposure amounts, and an 
apparent dynamic range of a video signal is increased 

15 by synthesizing one image from a plurality of images. 

In such a case, if a difference between a motion vector 
between pixels in a plurality of images and a motion 
vector between images is larger than a predetermined 
threshold value, the pixel is not used for 

20 synthesization in order to increase a substantial 

dynamic range. For example, even under rear light, not 
only the main object but also the background can have 
images at proper exposures . Even a video signal having 
an image shift caused by a motion of an object itself 

25 or hand vibration can be subjected to a practically 
usable dynamic range increasing process, to thereby 
prevent double or triple images . 
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Many widely different embodiments of the present 
invention may be constructed without departing from the 
spirit and scope of the present invention. It should 
be understood that the present invention is not limited 
to the specific embodiments described in the 
specification, except as defined in the appended 
claims . 
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WHAT IS CLAIMED IS: 

1. An image pickup method of increasing an 
apparent dynamic range of a video signal by 
synthesizing a single image from a plurality of images 
sequentially picked up at different exposure amounts, 
wherein a motion vector of a video signal is detected, 
and if the detection result indicates that the motion 
vector is larger than a predetermined threshold value, 
image synthesization is not performed. 

2. An image pickup method of increasing an 
apparent dynamic range of a video signal by 
synthesizing a single image from a plurality of images 
sequentially picked up at different exposure amounts, 
comprising: 

a motion vector detecting step of detecting a 
motion vector between corresponding pixels in the 
plurality of images; 

a vector difference detecting step of detecting a 
difference between a motion vector detected at said 
motion vector detecting step and a motion vector 
between the plurality of images; 

a comparison step of comparing the detection 
result of said vector difference detecting step with a 
predetermined threshold value; and 

a synthesization inhibiting step of inhibiting the 
image synthesization if "the comparison result at said 
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motion vector comparison step is larger than the 
predetermined threshold value. 

3. An image pickup method of increasing an 
5 apparent dynamic range of a video signal by 

synthesizing a single image from a plurality of images 
sequentially picked up at different exposure amounts, 
comprising: 

a motion vector detecting step of detecting a 
10 motion vector between corresponding pixels in the 
plurality of images; 

a vector difference detecting step of detecting a 
difference between a motion vector detected at said 
motion vector detecting step and a motion vector 
15 between the plurality of images; 

a comparison step of comparing the detection 
result of said vector difference detecting step with a 
predetermined threshold value; 

a coordinate converting step of performing a 
20 coordinate conversion of the plurality of images in 
correspondence with an image shift caused by a time 
difference when the plurality of images are picked up, 
if the comparison result at said motion vector 
comparison step is smaller than the predetermined 
25 threshold value; and 
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an image synthesizing step of synthesizing the 
plurality of images with the image shift corrected at 
said coordinate converting step, into a single image. 

5 4. An image pickup method according to claim 1, 

wherein a shutter speed is changed in order to change 
the exposure amount. 

5. An image pickup method according to claim 2, 
10 wherein a shutter speed is changed in order to change 

the exposure amount. 

6. An image pickup method according to claim 3, 
wherein a shutter speed is changed in order to change 

15 the exposure amount - 

7. An image pickup method according to claim 1, 
wherein an iris is changed at high speed in order to 
change the exposure amount. 

20 

8. An image pickup method according to claim 2, 
wherein an iris is changed at high speed in order to 
change the exposure amount. 

25 9. An image pickup method according to claim 3, 

wherein an iris is changed at high speed in order to 
change the exposure amount. 
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10. An image pickup apparatus in which an 
apparent dynamic range of a video signal is increased 
by synthesizing a single image from a plurality of 
images sequentially picked up at different exposure 
amounts, comprising: 

motion vector detecting means for detecting a 
motion vector of the video signal; 

comparison means for comparing the detection 
result by said vector difference detecting means with 
predetermined threshold value; and 

synthesization inhibiting means for inhibiting tt 
image synthesization if the comparison result by said 
comparison means is larger than the predetermined 
threshold value. 

11. An image pickup apparatus in which an 
apparent dynamic range of a video signal is increased 
by synthesizing a single image from a plurality of 
images sequentially picked up at different exposure 
amounts, comprising: 

motion vector detecting means for detecting a 
motion vector between corresponding pixels in the 
plurality of images; 

vector difference detecting means for detecting 
difference between a motion vector detected by said 
motion vector detecting means and a motion vector 
between the plurality of images; 
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comparison means for comparing the detection 
result by said vector difference detecting means with a 
predetermined threshold value; 

coordinate converting means for performing a 
5 coordinate conversion of the plurality of images in 
correspondence with an image shift caused by a time 
difference when the plurality of images are picked up, 
if the comparison result by said motion vector 
comparison means is smaller than the predetermined 
10 threshold value; and 

image synthesizing means for synthesizing the 
plurality of images with the image shift corrected by 
said coordinate converting means, into a single image. 

15 12. An image pickup apparatus according to claim 

10, wherein a shutter speed is changed in order to 
change the exposure amount. 

13. An image pickup apparatus according to claim 
20 11, wherein a shutter speed is changed in order to 

change the exposure amount. 

14. An image pickup apparatus according to claim 
10, wherein an iris is changed at high speed in order 

25 to change the exposure amount. 
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15. An image pickup apparatus according to claim 
11, wherein an iris is changed at high speed in order 
to change the exposure amount. 
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ABSTRACT OF THE DISCLOSURE 

An image pickup method and an image pickup 
apparatus are provided which can pick up an image with 
a broad substantial dynamic range even with a video 
5 camera or the like which is likely to be subject to 

image shifts. In increasing an apparent dynamic range 
of a video signal by synthesizing a single image from a 
plurality of images sequentially picked up at different 
exposure amounts, if a difference between a motion 

10 vector between pixels in a plurality of images and a 
motion vector between images is larger than a 
predetermined threshold value, the pixel is not used 
for synthesization. Even a video signal having an 
image shift caused by a motion of an object itself or 

15 hand vibration can be subjected to a practically usable 
dynamic range increasing process. 
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